Selecting Interventions that 
Succeed: Navigating Through 
Retention Literature 

Many developmental educators have been assigned 
increased responsibilities for campus-wide enrollment 
management activities. A new system is needed for 
educators to more accurately sort through the rapidly 
growing database of information related to student 
retention. This will enable the reader to more quickly 
David Arendale identify promising practices for further investigation. 

University of Minnesota 7^ article provides a scale to evaluate the likelihood 

of success among potential programs and identify 
them for further research. Scrutiny of potential re¬ 
tention programs must be increased by asking more 
questions early in the investigation process regarding: 
essential components of a program, research evalua¬ 
tion studies, barriers to successful implementation, 
and the availability of technical assistance to enable 
other institutions to successfully adapt and adopt the 
student retention practice. 

Developmental educators are increasingly called upon by their institutions 
to serve in positions of influence with enrollment management task forces 
that are charged with increasing student persistence and graduation rates. 
Being an expert with this topic has become more challenging as there has 
been an exponential growth in the professional literature associated with this 
topic. Several national conferences each year are devoted to student persis¬ 
tence (e.g., American Association of Collegiate Registrars and Admissions 
Officers, Noel-Levitz Centers). Two national publications publish in this 
area of scholarship (Journal of College Student Retention, www.baywood.com; 
Recruitment & Retention in Higher Education Newsletter, www.maganapubs. 
com). A number of publications identify best practices in this area (Habley 
& McClanahan, 2004; Noel, Levitz, & Saluri, 1985; Thomas, Quinn, Stack, & 
Casey, 2003; Upcraft, Gardner, & Barefoot, 2005). There are more than 4,000 
citations in the national ERIC database that contain the term student reten¬ 
tion in a postsecondary setting (ERIC, 2005). 

Sorting through all these conference presentations, reports, articles, books, 
brochures, and other descriptive literature is a challenge for any educational 
leader. Some of the literature describes home grown student retention pro¬ 
grams that have only operated at a single institution. Other publications 
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describe programs that have been replicated at other institutions in addition 
to the one that first created the academic intervention program. A better sys¬ 
tem is needed to help educators select from among this burgeoning database that 
all claim effectiveness for increasing student persistence and graduation rates. 

Traditional Selection System 

There are common patterns that many educators follow when seeking an 
intervention system to address the premature departure of students. Often 
a delegation of one or more is sent to a national conference to listen to a 
sample of concurrent presentations describing student retention programs. 
A decision about which session to attend is based on short presentation titles 
and 50-word summaries from the conference program book. Too often the 
speakers do not provide detailed research evaluation studies, detailed cost 
breakdowns, and barriers to implementation. A similar pattern is replicated 
in written reports or articles that describe the interventions. Little follow-up 
occurs between the speakers/authors. Institutions commit resources of time, 
personnel, and money to implementing intervention programs on the basis 
of several articles or listening to a conference presentation. A better system 
is needed to be more accurate, timely, and cost-effective in selecting new 
programs to adopt. 

New Selection System 

Selecting the appropriate academic intervention or making a change in insti¬ 
tutional policies is contingent upon many factors. What are the unique aca¬ 
demic and culture issues at the institution and which academic intervention (s) 
is best fitted to meet those needs? Is there clear evidence that the intervention 
will contribute to higher academic achievement at a particular institution 
as opposed to its past success at the institution where it was first developed? 
What is the capacity of the institution to implement the intervention or pol¬ 
icy regarding administrative support, faculty support, skill level of the inter¬ 
vention program, and the cost to implement and continue the program? 

Based on twenty years of experience as a learning center director, enroll¬ 
ment management leader, and director of the National Center for Supple¬ 
mental Instruction at the University of Missouri-Kansas City, I have de¬ 
veloped a different perspective for facilitating change regarding improved 
student graduation rates. An academic intervention or policy decision on one 
campus may not be effective on another one due to a variety of reasons. The 
following model presents a decision-making process that helps to more objec¬ 
tively enable institutions to compare among possible actions or intervention 
programs which are most appropriate for their particular situation and their 
capacity to enact change. 
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This student retention program model [Figure 1] has six stages of activities: 
(1) Identify the student retention problem by determining the characteristics 
of the students who are dropping out of the institution; (2) Sort through 
potential intervention programs by evaluating their likelihood of success at 
a particular institution; (3) Analyze the capacity of a particular institution 
to implement the intervention program; (4) Evaluate an institution’s campus 
climate and to what degree it will embrace and support the student subpopu¬ 
lation that it wants to retain; (5) Identification of specific action steps to be 
taken by an institution to change itself to be more supportive of students and 
conducive of their success; and (6) Implementation of the identified inter¬ 
vention program designed to increase student retention at the campus. 

Sorting Potential Interventions 

This article focuses on stage two of these activities: evaluating potential 
intervention programs regarding their likelihood of success at a particular 
institution. A prerequisite to selection of an academic intervention or making 
changes in campus policies is to carefully evaluate the evidence for effective¬ 
ness. Time limitations often preclude answering all of the following ques¬ 
tions during a conference presentation or addressing them during an article. 
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However it is reasonable for them to be addressed in professional articles, 
conference handouts, and during follow-up discussions with presenters. The 
following scale provides an increasing level of evidence of the likelihood of 
success in implementing the practice. Programs which reach the higher scale 
ratings are more likely to result in higher student persistence rates. 

1. There is little evidence or documentation that the practice has any evi¬ 
dence of effectiveness or is based on current research-based educational 
theory. This is the lowest rating that can be assigned to an interven¬ 
tion. In this case the conference presentation or article provides a basic 
description of the program, but no evidence of its effectiveness. With the 
rich professional literature that identifies other programs with evidence, 
spending much time investigating this intervention seems to be a waste 
of limited time resources. 

2. Practice is based upon sound educational theory and other previous¬ 
ly validated successful practices. Rather than only a description of the 
intervention, some explanations or theories are presented that suggest 
reasons for its effectiveness. What is the theory base for development of 
the intervention? Theory must lead practice, especially when considering 
the student population to be served. What is the connection between 
what we know about student departure (Tinto, 1993) and the interven¬ 
tion program? What other programs or practices similar to the one under 
scrutiny have been validated in the past? Most successful innovations are 
always based on elements of previously developed successful programs. 

3. Practice has undergone rigorous evaluation at one institution. What were 
the quantitative and/or qualitative research procedures employed? Are 
they rigorous and state-of-the art? Are they appropriate for the type of 
research questions being asked? Use of simple t-tests or student surveys 
is insufficient for research studies today. Rather than comparing to other 
research studies published, what are the new evaluation procedures be¬ 
ing discussed at national association conferences such as the American 
Educational Research Association? Does the program make claims that 
are unsubstantiated or overreaching? This is especially important if the 
practice is making claims related to student retention and graduation 
rates. 

The further the distance of time between when the intervention oc¬ 
curs and the outcome is measured, the less likely the impact. Too many 
intervening variables will have an impact upon the student in the mean¬ 
time. Such claims must be supported by highly sophisticated evaluation 
procedures. Does the research evaluation model take into account that 
the student has been impacted by multiples variables? Perhaps they have 
participated in simultaneous intervention programs? With hundreds of 
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variables having an impact upon students, the likelihood that only one 
intervention is responsible for the results is less likely. If the academic 
intervention model is a complex one with several activities operating si¬ 
multaneously, how has the research evaluation model identified the con¬ 
tributing impact of each separate activity upon the student outcomes? 
This is especially important if students have choices on which activities 
that they participate. 

Do the quantitative research procedures take into account background 
information about the individual students when conducting the analysis? 
Examples of this would include student demographics, affective domain 
(e.g., academic content mastery orientation, academic performance ori¬ 
entation, and self-efficacy), and academic preentry attributes as part of 
the research model. If the research model does not do so, the reported 
positive results may be due to the participation of students who were 
more academically prepared or motivated. 

What is the cost/benefit equation for the intervention? Careful factor¬ 
ing of all expenses associated with implementation is needed. For exam¬ 
ple, supervision and training costs are often underreported in conference 
presentations and journal articles. Often the most expensive component 
of an intervention is not the direct salaries, but the time required for 
activities both by the direct service provider and those who administer, 
supervise, and train (Arendale, 2001). The professional literature often 
cites these elements for many intervention programs to be effective. 
Maybe some optional activities are more helpful than others. Perhaps it 
is not necessary to implement all parts of a complex intervention model 
since only a few contribute the majority of the impact to the desired ef¬ 
fects. With tight budgets and limited resources, academic interventions 
will need to demonstrate their cost effectiveness as well as the improved 
student outcomes. 

4. Practice has undergone evaluation at one institution over a period of 
time with consistent results of positive outcomes. Have the research 
studies been carefully replicated over succeeding academic terms at the 
institution that originally developed the intervention? One of the most 
powerful research findings is the consistent report of positive outcomes 
over a long period of time. This helps to avoid the Hawthorne Effect of 
introducing something new in the environment and promoting short¬ 
term increased productivity before the environment returns back to the 
previous baseline behavior level. 

5. Practice has been validated by one or more external agencies (e.g., 
accrediting agencies, peer-reviewed publications, national awards com¬ 
petitions). What does a review of the professional literature suggest about 


5 


this or similar academic interventions for effectiveness? Publication of 
research about the intervention by a peer-reviewed journal increases 
the likelihood of effectiveness since it requires approval by an unbiased 
third-party editorial staff. Review by external accrediting agencies and 
national organizations conducting rigorous awards programs serve a 
similar purpose. 

6. Practice has been replicated successfully at several other institutions in 
addition to the one that originally created it. Has the academic inter¬ 
vention been successfully implemented at other institutions? Were the 
results replicated over succeeding academic terms? Were the institutions 
similar to the one considering its adoption? Were the students served at 
the other institutions similar to the one considering its adoption? An 
affirmative answer to these question increases the chance that the pro¬ 
gram can be successfully installed at another institution. This helps to 
address the potential problem that some programs are more dependent 
on the personality and traits of the original developer and less on the 
actual program components. In these cases, rarely is the program suc¬ 
cessful elsewhere. 

7. There are additional sources of information, consultation services, and 
training workshops about successful implementation of the practice. 
Will the institution or individual that developed the intervention pro¬ 
gram allow visitors to view the program in operation and talk with key 
individuals including the students who are served? Will it sell or share 
materials and consultation services? Considering the potential economic 
and social impact of implementation of a potential new program or pol¬ 
icy, it is critical to more fully understand the challenges with implemen¬ 
tation and ongoing operation. Few conference presenters have the time 
to share the challenges, failures, and details with program implementa¬ 
tion. Considering the total cost of starting and operating a new program, 
this is a small investment of time and money. 

Conclusion 

I am reminded of the expression, “we are drowning in data but are starved 
for knowledge and wisdom.” While we may have ready access to articles, 
reports, and presentation on student retention, we need to increase our scru¬ 
tiny of these information sources. We must ask more questions early in the 
investigation process, probe for the essential components of a program, and 
vigorously evaluate the research studies that evaluate the retention program. 
These activities will enable educators to more quickly and accurately sort 
through the confusing data and emerge with more likely prospects to enable 
our institutions to help students achieve their aspirations and dreams. 
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